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We discuss in detail the derivation of stochastic differential equations for the continuum time limit 
of the Minority Game. We show that all properties of the Minority Game can be understood by a 
careful theoretical analysis of such equations. In particular, jj we confirm that the stationary state 
properties are given by the ground state configurations of a disordered (soft) spin system; ii) we 
derive the full stationary state distribution; lii) we characterize the dependence on initial conditions 
in the symmetric phase and iv ) we clarify the behavior of the system as a function of the learning 
rate. This leaves us with a complete and coherent picture of the collective behavior of the Minority 
Game. Strikingly we find that the temperature like parameter which is introduced in the choice 
behavior of individual agents turns out to play the role, at the collective level, of the inverse of a 
thermodynamic temperature. 



I. INTRODUCTION 

Even under the most demanding definition, the Mi- 
nority Game (MG) [QJD definitely qualifies as a complex 
system. The MG can be regarded as an Ising model for 
systems of heterogeneous adaptive agents, who interact 
via a global mechanism that entails competition for lim- 
ited resource, as found for instance in biology and finan- 
cial markets. In spite of more than three years of intense 
research, its rich dynamical behavior is still the subject 
of investigations, many variations of the basic MG be- 
ing proposed, each uncovering new surprising regions of 
phase space. 

Most importantly, Refs. have shown that much 

theoretical insight can be gained on the behavior of this 
class of models, using non-equilibrium statistical physics 
and statistical mechanics of disordered systems. The ap- 
proach of Refs. |?|-[| rests on the assumption that, in a 
continuum time limit (CTL), the dynamics of the MG 
can be described by a set of deterministic equations. 
From these, one derives a function H which is minimized 
along all trajectories; hence, the stationary state of the 
system corresponds to the ground state of H, which can 
be computed exactly by statistical mechanics techniques. 
This approach has been challenged in Refs. which 
have proposed a stochastic dynamics for the MG, thus 
leading to some debate in the literature jl(],[ll]]. 

In this paper, our aim is to analyze in detail the deriva- 
tion of the CTL in order to clarify this issue. We show 
that a proper derivation of the CTL indeed reconciles the 
two approaches: the resulting dynamical equations - Eqs. 
( |l5| - |r7| ) below, which are our central result - are indeed 
stochastic, as suggested in Ref. but still the sta- 

tionary state of the dynamics is described by the minima 
of the function H, as suggested in Refs. [|[H]. We then 
confirm the analytic results derived previously. In few 
words, our analysis follows two main steps: first we char- 



acterize the average behavior of agents by computing the 
frequency with which they play their strategies. This step 
can be translated in the study of the ground state prop- 
erties of a soft spin disordered Hamiltonian. Secondly 
we characterize the fluctuations around the average be- 
havior. To do this, we explicitly solve the Fokker-Planck 
equation associated to the stochastic dynamics. 
The new results which we derive are: 

1. we derive the full probability distribution in the 
stationary state. Remarkably we find that the pa- 
rameter which is introduced as a temperature in 
the individual choice model, turns out to play the 
role of the inverse of a global temperature; 

2. for a > a c the distribution factorizes over the 
agents whereas in the symmetric phase {a < a c ) 
agents play in a correlated way. In the latter case, 
the correlations contribute to the stochastic force 
acting on agents. We show how the dependence of 
global efficiency on individual temperature found 
in Ref. || arises as a consequence of these correla- 
tions; 

3. we extend the analytic approach of Refs. ||,|[ to the 
a < a c phase and asymmetric initial conditions. 
The dependence on the initial conditions in this 
phase, first noticed and discussed in Refs. HQ], has 
been more recently studied quantitatively in Refs. 
HI?]]. We clarify the origin of this behavior and 
derive analytic solutions in the limit T — > 0. 

4. we show that the stronger is the initial asymmetry 
in agents evaluation of their strategies, the larger 
is the efficiency and the more stable is the system 
against crowd effects 

5. we derive the Hamiltonian of MGs with non-linear 
payoffs. 
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This leaves us with a coherent picture of the collective 
behavior of the Minority Game which is an important 
reference framework for the study of complex systems of 
heterogeneous adaptive agents. 



II. THE MODEL 

The dynamics of the MG is defined in terms of dynam- 
ical variables J7 Sj i(t) in discrete time t = 0, 1, . . . . These 
are scores, propensities or "attractions" [jl4| which each 
agent i = 1, . . . , N attaches to each of his possible choices 
s = 1, . . . , S. Each agent takes a decision Si(t) with 



Prob{s,(i) = s} = 



TiC/ s ,i(t) 



(1) 



where I\ > appears as an "individual inverse tempera- 
ture" . The original MG corresponds to I\ = oo jl| , and 
was generalized later to I\ = T < oo ||. 

The public information variable /i(t) is given to all 
agents; it belongs to the set of integers (1, . . . , P), and can 
either be the binary encoding of last M winning choices 
[Q| , or drawn at random from a uniform distribution [fl5f ; 
we stick to the latter case for sake of simplicity^]. The ac- 
tion OgS't) i of each agent depends the on its choice Si(t) 
and on fi{t). The coefficients i; called strategies, plays 
the role of quenched disorder: they are randomly drawn 
signs (Prob{a^ i = ±1} = 1/2), independently for each i, 
s and /i. On the basis of the outcome 



i=l 



(*)- 



each agent updates his scores according to 



u, ti (t + i) = u. ti (t) 



p 



(2) 



(3) 



The idea of this equation is that agents reward \U s ,i(t + 
1) > U Sl i(t)] those strategy which would have predicted 
the minority sign —A(t)/\A(t)\. The MG was initially 
proposed with a nonlinear dependence on A(t), i.e. with 



a dynamics U s ,i(t + 1) = U S) i(t) 



signA(t). This 



leads to qualitatively similar results. The extension of 
our theory to non-linear cases is dealt with in the ap- 
pendix. We shall not discuss any longer the interpreta- 
tion of the model, which is discussed at length elsewhere 

SW1- 

The source of randomness are in the choices of fi(t) by 
Nature and of Sj(t) by agents. These are fast fluctuating 



degrees of freedom. As a consequence also U s ^(t) and 
hence the probability with which agents chose Sj(i) are 
subject to stochastic fluctuations. Our analysis will in- 
deed focus on the characterization of the low-frequency 
fluctuations of U s j, by integrating our the high frequency 
fluctuations of fi(t) and Si(t). This will become clearer 
in the next section. For the time being let it suffice to 
say that there are two level of fluctuations, that of "fast" 
variables fi(t) and Si(t) and that of "slow" degrees of 
freedom U s ,i(t). 

The key parameter is the ratio a = P/N |l8| and two 
relevant quantities are 



= {A 2 ), H=^(A\tf 



(4) 



which measure, respectively, global efficiency and pre- 
dictabilitjjj]. 

Generalizations of the model, where agents account for 
their market impact J3],Q, where deterministic agents - 
so-called producers - are present , or where agents are 
allowed not to play |pO|-p^|, have been proposed. Rather 
than dealing with the most generic model which would 
depend on too many parameters, we shall limit our dis- 
cussion to the plain MG. Furthermore we shall specialize, 
in the second part of the paper to the case S = 2 which 
lends itself to a simpler analytic treatment. The analysis 
carries through in obvious ways to the more general cases 
discussed in Refs. @||||. 



III. THE CONTINUUM TIME LIMIT 



Our approach, which follows that of Refs. 
on two key observations: 



is based 



1. the scaling a 1 ~ N, at fixed a, suggests that typi- 
cally A(t) ~ v~N. Hence time increments of U St i(i), 
in Eq. (|) are small (i.e. of order VN/P - 1/VN); 

2. characteristic times of the dynamics are propor- 
tional to P. Naively this is because agents need 
to "test" their strategies against all P values of /i, 
which requires of order P time steps. More pre- 
cisely, one can reach this conclusion by measuring 
relaxation or correlation times and verifying that 
they indeed grow linearly with P (see Ref. ]Kj). 

The second observation implies that one needs to study 
the dynamics in the rescaled time r = t/P. This makes 



^Averages (. . .) stand for time averages in the stationary 

state of the process. Then (. . . l/x) stands for time averages 
*Both prescriptions lead to qualitatively similar results for , ,.\ 

^ * M j — i J conditional on fi(t) = fi. 

the quantities we study here. See [1161] for more details. 
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our approach differ from that of Refs. |§|,|9|], where the 
time is not rescaled. 

In order to study the dynamics for P, N 3> 1 at fixed 
a, we shall focus on a fixed small increment dr such that 
Pdr = aNdr ^> 1. This means that we take the con- 
tinuum time limit dr — > only after the thermodynamic 
limit N — > oo. We focus only on the leading order in N. 
Furthermore we shall also consider T, finite and 



Tidr < 1 



(5) 



which means that the limit I\ — > oo should be taken 
after the limit g?t — > 0. The orders in which these limits 
are taken, given the agreement with numerical simulation 
results, does not really matters: as we shall see differences 
only enter in the finite size corrections. We shall come 
back later to these issues. 

Iteration of the dynamics for Pdr time steps, from 
t = Pt to t = P(t + dr) gives 

P(r+dr)-l 

Us ,{T + dr)-u s ,{r) = -- < ( M*)' ( 6 ) 



t=Pr 



where we have introduced the functions u s ^(t) — 
U s ^Pt). 

Let us separate a deterministic (du s ^i) from a stochas- 
tic (dW s ,i) term in this equation by replacing 



a^ ) A(t) = a s>i (A)^ + X Sti (t). 



Here and henceforth, we denote averages over \x by an 
over-line 



R 



i p 



while (• . .)tt stands for an average over the distributions 



7r s ,i( T ) 



i P(r+dr)-l e r tUsAt) 



Pdr 



t=Pr 



(8) 



which is the frequency with which agent i plays strategy 
s in the time interval Pt < t < P(t + dr). 

Notice that 7r Sl j(r) will themselves be stochastic vari- 
ables, hence we also define the average on the stationary 
state as 



= lim — 

T„,T^OO T 



T +T 



dr(. 



t(t) 



(9) 



where the average inside the integral is performed with 
the probabilities 7r Sj , (r). 
Hence: 

U s ,i(r + dr) - u 8 ,i(r) = du s j(r) + dW S)i (r) 

P(r+dr)-l 

= -a fl , i (A) jr dT + - ^ X S;i (i) (10) 



Now the first term is of order dr as required for a deter- 
ministic term. In addition it remains finite as N — > oo 

II- 

The second term is a sum of Pdr random variables 
X s ^(t) with zero average. We take dr fixed and N very 
large, so that Pdr 3> 1 and we can use limit theorems. 
The variables X s ^(t) are independent from time to time, 
because both fi(t) and Sj(t) are drawn independently at 
each time. Hence X s ,i(t) for Pt < t < P(r + dr) are 
independent and identically^ distributed. For Pdr ^> 1 
we may approximate the second term dW s ^ of Eq. ( |l0| ) 
by a Gaussian variable with zero average and variance 



(dW Sti (T)dW rtj (T')) = 



8{t - t') 

P 2 



P(T+dr) 



(Xs^Xr^t))^ 



t=P-r 



p 

where the 8(r — r') comes from independence of X s ^(t) 
and X r j(t') for t ^ t' and the fact that X s ^(t) are iden- 
tically distributed in time leads to the expression in the 
second line. Now: 



(X S)i (t)X r> j(t)) ^ _ a Sii a rj j(A 2 ) v a s>i {A). K ar,j{A) n 



P 



P 



P 



(11) 



(7) The second term always vanishes for N — > oo because 



Sy i(A) 7T is of order iV° pq| . In the first term, instead, 



N 



(A 2 \v)„ = N+ Y, a s',k<'^s',kn r ,, l (12) 
fe^;=l s',r' 

is of order N which then gives a positive contribution in 
Eq. ( [ill) f or N -> oo. 

Eq. (f2|) leads to a stochastic dynamics where the noise 
covariance depends on the stochastic variables u S) ;(t) 
themselves. The complications which result from this 
fact can be avoided if one takes the approximation 



(A 2 \^ » (A*) = a 2 



(13) 



This approximation can be justified naively by observ- 
ing that the dependence on tt s ^ of the correlations only 
involves the global quantity (A 2 \fi), r /P for which one ex- 
pects some sort of self-averaging properties. Numerical 
results suggest that terms which are ignored by Eq. ( p^ ) 
are negligible for iV ^ 1, but we were unable to prove 



* Strictly speaking, Si(t) is drawn from the distribution in Eq. 
(^) and not from 7r Si i of Eq. (^). However these two distri- 
butions differ by a negligible amount as long as the condition 
(n) holds (see later). 



t=Pr 
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this in general |^(| (Eq. (|T^) holds trivially in the limit 

r -> o). 

Within this approximation, the correlation, for N ^> 1, 
become 

2 

(dW s 4T)dW rj (T')) = ^a-a-8{T - r')dr (14) 

Note that, for r ^ s or j ^ i, correlations 
(dW Sj i(T)dW ry j(T r )) oc a St ia r j ~ 1/VN vanishes as 
iV — > oo. However it is important to keep the off- 
diagonal terms because they keeps the dynamics of the 
phase space point \U(t)) = {f7 s ,i(*)}s=i,...,S, i=i,...,N con- 
strained to the linear space spanned by the vectors |a M ) = 
{a,g j} s =i,...,S, i=i,...,JV which contains the initial condition 
|C/(0)). The original dynamics of U s ^{t) indeed posses 
this property. 

It is important to remark that the approximation Eq. 
( |l3| ) makes our approach a self-consistent theory for a 2 . 
We introduce a 2 as a constant in Eq. (13) which then 



has to be computed self-consistently from the dynamic 
equations. 
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FIG. 1. Noise strength averaged over all agents (stars); 
when it is multiplied by a (diamonds), one recovers a' 2 /N 
(circles) (P — 32, S = 2, 300P, iterations, F = oo, average 
over 50 samples). Dashed lines are for eye guidance only 

Summarizing, the dynamics of u Sl i is described by a 
continuum time Langevin equation: 



du s ,i(T) 



dr 



= -a Sti (A) + J? s ,i(r) 



(15) 
(16) 



confirm Eq. ( |l7| ) is valid even for a < a c . Fig. || com- 
pares the results of numerical simulations of the MG, as 
a function of T, with those of a semi-analytic solution of 
Eqs. ( P^HTt| ), to be discussed later. The agreement of the 
two approaches shows that Eqs. (p^[-p7|) are valid even 
in the symmetric phase (a < a c ) for all values of T. 

The instantaneous probability distribution of s, in the 
continuum time limit, reads 



T 8 ,i(r) 



i(r) 



£ re rw, ( (r)- 



(18) 



This and Ito calculus then lead to a dynamic equation 
for 7r Sl i(r). We prefer to exhibit this for I\ = T and using 
the rescaled time t = Ft: 



dir s 



At 



C7 2 T 

aN' 



a Sli {A) - TTi ■ di{A) 

,i {^s,i - k 2 ) + VT7r s ,i (r] s 



nffi). (19) 



The first term in the r.h.s. comes from the deterministic 
part of Eq. (plj), the second from the Ito term (where 
we neglected terms proportional to ai, s a>i,s' ~ l/V^/V for 



s ^ s') and the third from the stochastic part. It is clear 
that, in the limit r — > the last two term vanish and the 
dynamics becomes deterministic. 

We see then that V tunes the strength of stochastic 
fluctuations in much the same way as temperature does 
for thermal fluctuations in statistical mechanics. The 
"individual inverse temperature" T should indeed more 
correctly be interpreted as a learning rate^]. Furthermore 
it plays the role of a "global temperature" . We shall 
pursue this discussion in detail below for the case S = 2. 

At this point, let us comment on the limit r — > oo, 
which is of particular importance, since it corresponds 
to the original MG. It is clear that in the limit r — > oo 
the dynamical equations (^|) become problematic. The 
origin of the problem lies in the order in which the lim- 
its N — > oo and T — > oo is performed. Indeed in Eq. 
(|) U„ ti (t) ~ u Sti (r) + 0(dr) for Pt < t < P(r + dr). 
Therefore, as long as Tdr <C 1 the difference between 
Eq. (N) and t: s ^ in Eq. jl^ ) is negligible. In practice, in 
order to satisfy both Tdr <C 1 and Pdr ^> 1, one needs 
r <C P. When this condition is not satisfied the instanta- 
neous probability Eq. (Q) fluctuates very rapidly at each 
time-step. Eq. (||) averages out these high frequency 
fluctuations so that, even for T = oo, the distribution 
K s .i{ T ) °f Eq. (1q) is not a discontinuous step function of 



{Va,i( T )Vr,j(T)) = —a s ,ia r jS(r - r'). (17) 

Equation (^), given its derivation, has to be inter- 
preted in the Ito sense. The expression for the noise 
strength is confirmed by figure [I] where the measure of 
X = E 2 , s ((^ s >/(™ 5 ) in a MG is reported; note that 
these numerical simulations were done for r = oo and 



§This is an a posteriori learning rate. Indeed 1/F is the time 
the dynamics of the scores needs in order to learn a payoff 
difference. From a different viewpoint, V tunes the random- 
ness of the response of agents. The larger the randomness, 
the longer it takes to average fluctuations out. 
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U Sl i(r), as suggested by Eq. (|18|). High frequency fluc- 
tuations contribute to the functional form of Ti St i on u St i 
which will differ from Eq. (|l8|). 

Summarizing, when we let Y — > oo only after the limit 
N — > oo has been taken no problem arises. There is 
no reason to believe that results change if the order of 
the limits is interchanged. This expectations, as we shall 
see, is confirmed by numerical simulations (see Fig. |^): 
direct numerical simulations of the MG deviate from the 
prediction of Eqs. (p^-p7|) only for finite size effects which 
vanish as N — > oo. 

Eqs. ( p^[]l7| ) are our central result. We shall devote the 
rest of the paper to discuss their content and to show that 
all of the observed behavior of the MG can be derived 
from these equations. 



IV. STATIONARY STATE 

Let us take the average, denoted by (...), of Eq. flU) 
on the stationary state (SS). Let 

fs,i = (7T a ,i) 

be the frequency with which agent i plays strategy s in 
the SS. Then we have 



dU s , 
dr 



-a.,i(A), (A\^) = J2fs',j< tj 



Given the relation between tt s ^ and U s ^ and considering 
that the long time dynamics of U a i in the SS is U St i{r) = 
const+v S! iT, we have that i) each strategy which is played 
in the SS by agent i must have the same "velocity" v St i = 
v* , and ii) strategies which are not played (i.e. with 
fs.i — 0) must have v s j < v* . In other words 



-a St i(A)=v*, V«,s such that f s ,i>0 (20) 
-a s ,i(A) < v*, V«,s such that f Sji = 0. (21) 

Consider now the problem of constrained minimization 
of H in Eq. ([|), subject to f Si i > for all s, i and the nor- 
malization conditions. Introducing Lagrange multipliers 
Xi to enforce fs.i = 1 f° r a ^ h this problem reads 



min <(A} 2 - 

Us,*>o} 



N 



>('-£'-)} 



(22) 



Taking derivatives, we find that if f S i > then 



a s ,i(A) + Xi = whereas if f s i = then a Sj i(A) + A; > 0. 
These are exactly Eqs. d20| , |2l"| ) where v* = A^. We then 
conclude that the two problems Eqs. (20 2l]) and Eq. 



( f22| ) are one and the same problem^. In other words f s ^ 
can be computed from the constrained minimization of 
H as proposed in Refs. H,||. 

Hence the statistical mechanics approach based on the 
study of the ground state of H is correct. This ap- 
proach gives the frequency f s ^ with which agents play 
their strategies. 

We remark once more that H is a function of the sta- 
tionary state probabilities f St i. Also note that 



H{TTsA 



N 

E 

— l s,s' 



E 



as a function of the instantaneous probabilities 7r Si i, is 
not a Lyapunov function of the dynamics. The dynami- 
cal variables ir s ^(t) are subject to stochastic fluctuations 
of the order of y/Tl around their average values f s ^. Only 
in the limit F; — > 0, when the dynamics becomes deter- 
ministic and 7r Si i — > / S) j, the quantity H{n St i} becomes a 
Lyapunov function. 

The solution to the minimization of H reveals two qual- 
itatively distinct phases |3|Q] which are separated by a 
phase transition occurring as a — > a c . We discuss quali- 
tatively the behavior of the solution for a generic S and 
leave for the next section a more detailed discussion in 
the simpler case 5 = 2. 



A. Independence on T for a > a c 

When a > a c the solution to Eq. ( |22] ) is unique and 
H > 0. Hence f S: i does not depends on F, neither does 
H . In addition we shall see that 

(^s.i^s'j) = (7r s ,i)(7iyj) = f s sfs>,] for i ^ j (23) 
implying that 



- 2 -^+EE 



ijtj s,r 



does not depend on T cither. Hence the solution {f s .i} 
uniquely determines all quantities in the SS, as well as 



"Indeed both problems can be put in the form of a Linear 
Complementarity problem p7| : 

fs.i > 
as, t a a >, jfs'.j +v* > 



This problem has a solution for all values of v* because of 
non-negativity of the matrix TT^Ja^TJ, see Ref. [E7j . 
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the parameters which enter into the dynamics (notice the 
dependence on a 2 in Eq. |l7| ). In particular a 2 does not 
depend on T. 



B. Dependence on V and on initial conditions for 

a < a c 

For a < a c the solution to the minimization problem 
is not unique: there is a connected set of points {f s ,i} 
such that H — 0. Let us first discuss the behavior of the 
system in the limit r — > 0, where the dynamics becomes 
deterministic. The dynamics reaches a stationary state 
{fs,i} which depends on the initial conditions. 

In order to see this, let us introduce the vector notation 
|«) = s = l,...,S,i = l,...,N}. Then for all 

times \u(t)) is of the form 

p 

\u(t)) = \u(0))+J2\*W 

where C^(t) are P functions of time. 

If there are vectors (v\ such that (v\a^) = for all 
/U, then (v\u(t)} = (v\u(0)}, i.e. the components of the 
scores will not change at all along these vectors. As a re- 
sult the SS will depend on initial conditions \u(0)). These 
vectors (v\ exist exactly for a < a c because the "di- 
mensionality" of the vectors \u(t)) is larger than Ff^. 

The picture is made even more complex by the fact 
that, for a < a c , when T is finite Eq. (|2^) does not 
hold. Hence a 2 has a contribution which depends on the 
stochastic fluctuations around f s ^. The strength of these 
fluctuations, by Eqs. (|l7|,[l9]), depends on T and a 2 itself. 
We face, in this case, a self-consistent problem: a 2 enters 
as a parameter of the dynamics but should be computed 
in the stationary state of the dynamics itself. Therefore 
the solution to this problem and hence a 2 depends on 
r. The solution {f s ,i} to the minimization of H should 
also be computed self-consistently. As a result, the SS 
properties acquire a dependence on T. 

The condition (|23|), which is similar to the clustering 
property in spin glasses Q , plays then a crucial role. We 
show below how the condition (f23|), the dependence on 
initial conditions and on T enter into the detailed solution 
for S — 2. By similar arguments our conclusion can be 
generalized to all S > 2. 



"in order to compute the dimensionality of the vectors |m) 
we have to take into account the N normalization conditions 
and the fact that strategies which are not played = 0) 
should not be counted. So if there are N > variables f s .i > 0, 
the relevant dimension of the space of |u) is iV> — N. Hence 
vectors (v| orthogonal to all |a M ) exist for iV> — N > P, i.e. 
for a < a c = iV>(a c )/iV - 1. 



V. THE CASE S = 2 

We work in this section with the simpler case of S = 2 
strategies, labelled by s — ±. We also set I\ = T for all 
i. Following Refs. P,[13| we introduce the variables 



U Li 



Let us rescale time t = Tt and introduce the variables 

B(t) = r "+..M-"-M 

Then, using Eq. (||), the dynamical equations ( p^jTi] ) 
become 

= -&n - £ tej tanh^ ) + Ci (24) 



(25) 



The Fokker-Planck (FP) equation for the probability 
distribution P({yi},t) under this dynamics reads 

dPUmht) = £ 9 f_ + ^_ tanh(tf _ )+ 



at 



, N „ ) 

y ^ i ^Wi\ Pi{Viht) (26) 



where we have introduced the parameter 

2aN 



Ta 2 



(27) 



Multiply Eq. (g6|) by yi and integrate over all variables. 
Using integration by parts, assuming that P — > fast as 
y.j —>■ oo, one gets 



d_ 

Of 



N 



(Vi) = (tanh(yj)) ■ 

3=1 



Let us look for solutions with (yi) ~ Vit and define 
mi = (tanh(?/i)). Hence for t — » oo we have 



JV 



3=1 



(28) 



Now, either Vi — and (yi) is finite, or vi ^= 0, which 
means that j/, —> ±oo and rrii — sign^j. In the latter 
case (vi ^ 0) we say that agent i is frozen jljj, we call 
T the set of frozen agents and 4> = \F\/N the fraction of 
frozen agents. 



G 



As in the general case, the parameters Vi for i £ T and 
rrii = (tanh(y^)) for i $jL T are obtained by solving the 
constrained minimization of 

2 



H = 



-t 

P ^ 



IF 



N 



When the solution of minif is unique, i.e. for a > a c , 
the parameters rrii depend only on the realization of dis- 
order and their distribution can be computed 
as in Ref [||. When the solution is not unique, i.e. for 
a < a c , we are left with the problem of finding which 
solution the dynamics selects. Let us suppose that we 
have solved this problem (we shall come back later to 
this issue), so that all rrii are known. 

Using the stationary condition Eq. fl28|), we can 
write the FP equation for the probability distribution 
Pu(yi, i & F) of unfrozen agents. For times so large that 
all agents in J- are indeed frozen (i.e. Si(t) — signup) this 
reads: 

dP u _ y> d 

at -fed* 



-or = Yl «r- Y titj tanh (y. 



-—>r 



This has a solution 

P u oc exp < -^y^ [log cosh yj 



(29) 



Finally we have to impose the constraint that \y(t)) = 
{Vi(t)}iLi must lie on the linear space spanned by the 
vectors which contains the initial condition |y(0)). 
This means that 



Pu oc V y (Q) exp I -(5 ^2 [log cosh - m 3 y 3 \ 
where the projector V v (q) ^ s given by 



P r oc n 

fJ ,=x J -° i=l 



(30) 



(31) 



We find it remarkable that T, which is introduced as 
the inverse of an individual "temperature" in the defini- 
tion of the model, actually turns out to be proportional 
to (see Eq. |27]) which plays collectively a role quite 
similar to that of temperature. 



Using the distribution Eq. (30), we can compute 



N 



- 2 = tf+Ec 2 a 



i=l 



E&£j(( taJ1 ky» _ TOi)(tanh?/j - rrij)). 

i¥=3 



(32) 



This depends on /3, i.e. on a 2 itself by virtue of Eq. 
( p7j ) . The stationary state is then the solution of a self- 
consistent problem. Let us analyze in detail the solution 
of this self-consistent problem. 



A. a > a c 

For a > a c the solution of mini? is unique, and hence 
rrii depends only on the realization of the disorder. In 
addition, the number N — \T\ = N(l — <fi) of unfrozen 
agents is less than P and the constraint is ineffective, 
i.e. V y (o) = 1. The scores \y) of unfrozen agents span a 
linear space which is embedded in the one spanned by the 
vectors Hence the dependence on initial conditions 
yi(0) drops out. Therefore the probability distribution 
of yi factorizes, as in Eq. (29). Then the third term 
of Eq. ([52j), which is the only one which depends on (3, 
vanishes identically. We conclude that, for a > a c , a 2 
only depends on m, and is hence independent of L as 
confirmed by numerical simulations. 

Summarizing, for a > a c one derives a complete solu- 
tion of the MG by finding first the minimum {rrii} of H 
and then by computing a 2 , (3 and the full distribution of 
yi from Eq. @. 



B. a < a c : dependence on V and crowd effects 

When a < 1 — <\> 1 on the other hand, the solution of 
min H is not unique. Furthermore the constraint cannot 
be integrated out and the stationary distribution depends 
on the initial conditions. 

Numerical simulations S show that a 2 increases with 
r for a < a c (see Fig. ||). This effect has been related 
to crowd effects in financial markets Ref. j|] has 

shown that crowd effects can be fully understood in the 
limit a — > 0: as T exceeds a critical learning rate r c , the 
time independent SS becomes unstable and a bifurcation 
to a period two orbit occurs. Neglecting the stochastic 
term Q, Ref. H shows that this picture can be extended 
to a > OR. This approach suggests a crossover to a 



**The idea of Ref. |j| is the following: imagine that our 
system is close to a SS point y* at time t = tk, when y(t) 
I Will the system be close to y* when the pattern y, — 1 
occurs again the next time t' — tfc+i? To see this, let us 
integrate Eq. from t = tk to tk+i- In doing this we 

i) neglect the noise term (i.e. Q — 0) and ii) assume that 
tanh?/i(t) w tanh yi(tk) stays constant in the integration time 
interval. This latter assumption is similar to the recently 
introduced [^| batch version of the MG, where agents update 
their strategies every P time-steps. This leads to study a 
discrete time dynamical system 



Vi(th+i) = y%{tk) 



N 

+ ^ 55 tanh[?/j (t k ) 
3=1 



(33) 



where the factor F comes because tk+i—tk is on average equal 
to r. The linear stability of fixed point solutions is analyzed 
setting y%(tk) = yt + Syi(k) and computing the eigenvalues of 
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"turbulent" dynamics for T > r c , where 

4 



r c (a) = 



(1 + ^) 2 (1-Q) 



(34) 



and 



1 



N 



i=l 

Both Q and T c can be computed exactly in the limit 
TV — > oo within the statistical mechanics approach . 

This approach however i) does not properly takes into 
account the stochastic term, ii) does not explain what 
happens for r > T c and Hi) does not explains why such 
effects occur only for a < a c . 
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FIG. 2. Global efficiency a 2 /N versus r for a = 0.1 < a c 
and different system sizes. Lines refer to direct simulations 
of the MG with TV = 160, 320 and 640. Finite size effect for 
r> 1 are evident. Symbols refer instead to the solution of the 
self-consistent equation (^) for the same system sizes. For 
both methods and all values of TV, a 2 is averaged over 100 
realizations of the disorder. The arrow marks the location 
of r c predicted by Eq. (Q). In the inset, the theoretical 
prediction Eq. (B7|) on the leading behavior of a 2 /N for K 1 
(solid line) is tested against numerical simulations of the MG 
(points) for the same values of TV. 

The stochastic dynamics derived previously gives de- 
tailed answers to all these issues. We first restrict atten- 
tion to symmetric initial conditions yi(0) — and then 
discuss the dependence on initial conditions. The choice 
of j/i(0) = Vi is convenient because it allows one to 
use this same symmetry to identify the solution {nii} to 
the minimization of H, independently of T. To be more 
precise, one can introduce a "magnetization" 



the linearized map Syi(k+1) ~ ^ . Ti t jSyj(k). There is acrit- 
ical value of F above which the solution y* become unstable, 
which is given by Eq. (pij). 



1 N 1 N 

i=l i=l 



(35) 



which measures the overlap of the SS configuration 
with the initial condition. Symmetric initial conditions 
are related to M = SS. These are the states we focus 
on. The solution is derived in two steps: 

1. find the minimum {m;} of H, with M = 0; 

2. compute self-consistently a 2 . 

The numerical procedure for solving the problem is the 
following: given the realization of disorder f2 M }, step 
(1) — finding the minimum {mj of if — is straightfor- 
ward. For step (2) we sample the distribution Eq. (|3p| ) 
with the Montecarlo method^ at inverse temperature /3 
and measure the (3 dependent contribution of a 2 in Eq. 



EC9) = X;^<(tanhy i 



TOi)(tanhyj - rrij))p. 



Here < 
Eq. f| 



.)/3 stands for an average over the distribution 
with parameter (3. Finally we solve the equation 



cr 2 (r) = cr 2 (0) + S 



2aTV 

iv 2 (r) 



(36) 



This procedure was carried out for different system 
sizes and several values of T. The results, shown in Fig. 
^, agree perfectly well with direct numerical simulations 
of the MG. Actually Fig. | shows that, for T > 1, the 
solution of the self-consistent equation ( |36| ) suffers much 
less of finite size effects than the direct numerical simu- 
lations of the MG. Fig. || also shows that, even if only 
approximate, Eq. (|34|) provides an useful estimate of the 
point where the crossover occurs. 

It is possible to compute a 2 (r) to leading order in V <C 
1. The calculation is carried out in the appendix in detail. 
The result is 



<7 

TV 



l-Q 



1 



l-Q + q(l-3Q) 
4a 



r + o(r 2 ) 



(37) 



"The Montecarlo procedure follows the usual basic steps: i) 
A move yi — > yt + e£f is proposed, with and e drawn at 
random, ii) the "energy" 



N 



E{yi} = [log cosh yt - rrnyi 



of the new configuration is computed and Hi) The move is 
accepted with a probability equal to min(l, e~ l3AE ) where AE 
is the "energy" difference. 
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The inset of Fig. |2| shows that this expression indeed 
reproduces quite accurately the small T behavior of a 2 . 
Note finally that Eq. (|3^) has a finite solution a 2 (oo) = 
cr 2 (0) + S(0) in the limit r — > oo. Furthermore it is easy 
to understand the origin of the behavior a 2 /N ~ 1/a for 
Eq. (p6|). Because of the constraint, when is positive 
(negative) the fluctuations of tanh(j/j) — are positively 
(negatively) correlated with tanh(j/j) — mj. If we assume 
that ([tanh(yi) — TO,][tanh(j/j) — TOj]) ~ c£j£j for some 
constant c, we find S ~ c X^j • This leads easily 
to E/JV ~ cj (4a), which explains the divergence of a 2 /N 
as a -y for T > 1. 

C. Selection of different initial conditions in the 
Replica calculation 

As discussed above, the stationary state properties of 
the MG in the symmetric phase depend on the initial 
conditions. Can the statistical mechanics approach to the 
MG be extended to characterize this dependence for 
r <C 1? If this is possible, how do we expect the resulting 
picture to change when T increases? We shall first focus 
on the first question (i.e. r « 1) and then discuss the 
second. 

Of course one can introduce the constraint on the dis- 
tribution of j/i in the replica approach in a straightfor- 
ward manner. This leads however to tedious calculations. 
We prefer to follow a different approach. In the a < a c 
phase the minimum of H is degenerate, i.e. H = occurs 
on a connected set of points. Each of these points corre- 
sponds to a different set of initial conditions, as discussed 
above. In order to select a particular point with H = 
we can add a potential r\ J2i( s i ~ s i ) 2 /2 to the Hamilto- 
nian H, which will favor the solutions closer to s*, and 
then let the strength rj of the potential go to zero. This 
procedure lifts the degeneracy and gives us the statistical 
features of the equilibrium close to s* . 

The nature of the stationary state changes as the 
asymmetry in the initial conditions changes. If we take 
s* = s* , the state at s* — describes symmetric initial 
conditions and increasing s* > gives asymmetric states. 

The saddle point equations of the statistical mechanics 
approach of Ref. || can be reduced to two equations: 

/oo 
Dzs 2 {z) (38) 

1 + V f°° 
X = . = * / Dzz So (z) (39) 
y/a(l + Q) J -00 

where Dz = -^=e~ z2 / 2 , s (z) £ [—1, 1] is the value of s 
which minimizes 

V,{s) = \s 2 - sj^-zs + + X )(s s*) 2 (40) 



and x = fl(Q — q)/oi is a "spin susceptibility". There are 
two possible solutions: one with x < 00 finite as 77 — > 
which describes the a > a c phase. The other has X ~ I/ 7 ? 
which diverges as r\ — > 0. This solution describes the 
a < a c phase. We focus on this second solution, which 
can be conveniently parameterized by two parameters Zq 
and eo- We find 

!-l if z < —zq - eo 

if -zo - e a < z < z - e 
1 if z > z - e 

Indeed Eq. (|38|) gives Q(z ,eo) and Eq. (|39|) which 
for x ~~ * 00 reads \J a(l + Q) = J Dzzso(z), then gives 
a(za, e ). 

With eo 7^ one finds solutions with a non-zero "mag- 
netization" M — (sj). This quantity is particularly 
meaningful, in this context, because it measures the over- 
lap of the behavior of agents in the SS with their a priori 
preferred strategies 

M= D** (z)= lim -5>i(*)*(0)>. (41) 

Note indeed that one can always perform a "gauge" 
transformation in order to redefine s — +1 as the initially 
preferred strategy. This amounts to taking yi(0) > for 
all i. 

Which SS is reached from a particular initial condition 
is, of course, a quite complex issue which requires the in- 
tegration of the dynamics. However, the relation between 
Q and M derived analytically from Eqs. d38| , [4l| ) can eas- 
ily be checked by numerical simulations of the MG. Fig- 
ured shows that the self-overlap Q and the magnetization 
M computed in numerical simulations with initial con- 
ditions yi(0) = yo for all i, perfectly match the analytic 
results. The inset of this figure shows how the final mag- 
netization M and the self-overlap Q in the SS depend on 
the asymmetry yo of initial conditions. 




0.2 0.4 0.6 0.8 
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FIG. 3. Relation between Q and M, for a = 0.1, derived 
from analytic calculation (full line) and from numerical sim- 
ulations of the MG with different initial conditions yo (o, 
P = 32, N = 320, T = 0.1). The inset shows the depen- 
dence of Q and M from the initial condition yo- 

Let us finally discuss the dependence on T for asym- 
metric initial conditions. Eq. ( |34| ) provides a character- 
istic value of r as a function of a and Q. This theoretical 
prediction is tested against numerical simulations of the 
MG in Fig. [l| when plotted against r/r c , the curves 
of a 2 /N obtained from numerical simulations, approxi- 
mately collapse one onto the other in the large T region. 
Fig. ^ suggests that T c in Eq. ([54]) provides a close lower 
bound for the onset of saturation to a constant a 2 for 
large T. We find it remarkable that a formula such as 
Eq. (34) which is computed in the limit V — > 0, is able 



to predict the large T behavior. 
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FIG. 



Global 



efficiency 



a 2 /N versus r/r c for a — 0.1 < a c , N = 160 agents and 
different initial conditions yo- The value F c is computed from 
Eq. (|34|) and is shown in the legend. 

With respect to the dependence on initial conditions, 
we observe that r c is an increasing function of Q and 
hence it increases with the asymmetry yo of initial con- 
ditions. Hence, for a fixed T, the fluctuation dependent 
part £ of a 2 decreases with yo because it is an increasing 
function of r/r c . This effects adds up to the decrease of 
the r independent part of a 2 discussed previously. 

Fig. |i| also shows that the r>r c state is independent 
of initial conditions yo. This can naively be understood 
observing that stochastic fluctuations induce fluctuations 
8yi which increase with V. For r > 1 the asymmetry 
yo of initial conditions is small compared to stochastic 
fluctuations Syi and hence the system behaves as if yo ~ 
0. 



D. The maximally magnetized stationary state 

The maximally magnetized SS (MMSS), obtained in 
the limit yo — * oo, is also the one with the largest value 
of Q, and hence with the smallest value of a 2 = N(l — 
Q)/2. a 2 /N is plotted against a both for symmetric 
yo = initial conditions and for maximally asymmetric 
ones yo — > oo in fig. |5|. The inset shows the behavior of 
Q and M in the MMSS. 



' 'e 




FIG. 5. a 2 /N for the MG with initial conditions y = 
(full line) and yo — + oo (dashed line). The inset reports the 
behavior of M and Q in the yo — > oo SS. Note that Q is linear 
in a. 

Remarkably we fi nd that a 2 /N vanishes linearly with 
a in the MMSSp*l This means that, at fixed P, as N 
increases the fluctuation a 2 remains constant. This con- 
trast with what happens in the yo = state, for T <C r c , 
where a 2 increases linearly with N, and with the case 
T > T c where a 2 oc N 2 |[|,|] . Note also that the lowest 
curve of Fig. ^| also gives an upper bound to the a 2 of 
Nash equilibria (see Refs. |[|jj§j]). 

The MMSS is also the most stable state against crowd 
effects: if we put Q(a, yo = oo) = 1 — ca, as appropriate 
for the MMSS we find that r c ~ 1/a diverges with a. 



VI. CONCLUSIONS 

We have clarified the correct derivation of continuous 
time dynamics for the MG. This on the one hand recon- 
ciles the two current approaches [§J|]. On the other it 
leads to a complete understanding of the collective be- 
havior of the MG. We confirm that stationary states are 
characterized by the minimum of a Hamiltonian which 



***This results was also found analytically in 
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measures the predictability of the game. For a > a c we 
find a complete analytic solution, whereas for a < a c 
the statistical mechanics approach of Ref. |3| is valid for 
r — > 0. It is in principle possible to introduce the new 
elements discussed here in the approach of Ref. ||] and 
to derive a full analytic solution. We have indeed derived 
the first term of the series expansion for r -C 1, which 
agrees perfectly with numerical data. The extension of 
the approach of Ref. || involves lengthy calculations and 
it shall be pursued elsewhere. 

Finally we note that the results derived in this paper 
generalize to more complex models. It is worth to remark 
that the solution to the FP equation is no more factor- 
izable, in general, when agents account for their market 
impact as in Refs. Hence, as long as there are 

unfrozen agents, we expect that the stationary state de- 
pends on r. However, when the agents take fully into 
account their market impact, all of them are frozen and 
the conclusion that agents converge to Nash equilibria 
remains valid. 

We acknowledge constructive discussions with A.C.C. 
Coolen, A. Engel, J. P. Garrahan, J.A.F. Heimel and D. 
Sherrington. 

APPENDIX A: NON-LINEAR MINORITY 
GAMES 

Take a generic dynamics 

U s ,i(t + 1) = U s At)-a»fg[A(t)] 

with g(x) some function. When we carry out the limit 
to continuous time we find a deterministic term which is 
proportional to —a s ,i(g(A)). The stationary state condi- 
tions then read 

Vi = -a S) i(g(A)), if/ s ,i>0 

and 

Vi > —a a ,i(g(A)}, if f s j = 0. 

For any fixed fi, A(t) is well approximated by a Gaus- 
sian variable with mean 

i,s 

and variance D = a 2 — H. Here we neglect dependences 
on /i. Also we treat D as a parameter and neglect its de- 
pendence on the stationary state probabilities / Sl j. Hence 
we can write 

(g(AM = |~ ^- x2/2 9 (0%) + ^) 

The stationary state conditions of the dynamics above 
can again be written as a minimization problem of the 
functional 




and D = a 2 — H which must be determined self- 
consistently. 

Indeed taking the derivative of H g w.r.t. f Si i and im- 
posing the constraint f s ^ > and normalization, we ar- 
rive at exactly the same equations which describe the 
stationary state of the process. 

The Hamiltonian for the original MG is derived setting 
g(x) = signx, which leads to 

^"p^k + vd \vd ) : 

The analysis of stochastic fluctuations can be extended 
to non-linear cases in a straightforward manner. Again 
the key point is that the dynamics is constrained to the 
linear space spanned by the vectors \a^). For a > a c we 
have no dependence on initial conditions. However it is 
not easy to show in general that the distribution of scores 
factorizes across agents. This means that there may be 
a contribution of fluctuations to a 2 - i.e. S > - so 
we cannot rule out a dependence of a 2 on T. Numerical 
simulations for g{x) = signx show that such a depen- 
dence, if it exists, is very weak. Anyway even though 
a 2 only depend on / S) j, the minimization problem de- 
pends on D = a 2 — H which must then be determined 
self-consistently. 

For a < a c the dependence on initial conditions in- 
duces a correlation of scores across agents. As a result 
a 2 depends on T just as in the linear case discussed above. 

APPENDIX B: SMALL V EXPANSION 

For T 1 it is appropriate to consider (3 ^> 1 and to 
take 

Hi = arc tanh mi H — j= 

so that /Jflogcoshy* — m t y^\ ~ |(1 — m 2 )z 2 + O^ 1 / 2 ). 
Hence we have to sample a distribution 

P{ Zi } oc e- L ^- m >' 
where Zj has the form 
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It is convenient to express everything in terms of the 
coefficients c M . Their pdf is derived from that of Zi and 
it reads: 

N 
i=l 

From this we find (cPc") = [T^ 1 ]^- 

Now we split the term S(/3) in two contributions, 



N 



y^^(tanh^ - m,-) 



N 



J2^f [ml - (tanhyif 



and work them out separately. For the first we use 
- 1 



so that 



JV 



^^(tanhy 4 - m{) 



i=l 



(3P 

TtT _ 1-Q 
~W ~ 2/3 



AT. 



Within the approximation (1 — 3m?) « (1 — 3Q) we 
are able to derive a closed expression also for the second 
term: 



JV 1 N 

^fKMtanh^) 2 ] = -£^(l-m?)(l-3m?)< 



Hence we find 



S(/3) 



1 



al-3Q 
1-3Q 



AT. 



2 2 
This and Eq. (|27|) lead to Eq. (j 



P 



o(r 2 ). 
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